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A systematic theory is introduced that describes stochastic effects in game theory. In a biological 
context, such effects are relevant for the evolution of finite populations with frequency-dependent 
selection. They are characterized by quantum Nash equilibria, a generalization of the well-known 
Nash equilibrium points in classical game theory. The implications of this theory for biological 
systems are discussed in detail. 



I. INTRODUCTION 

Classical game theory is a well-known mathematical 
formalization of competitions with rational rules and ra- 
tional players jjj. The strength of this theory lies in 
the abstraction from the detailed scenario. In its sim- 
plest form, a game is reduced to a set of basic strategies 
i = 1, . . . , s and a matrix A = (Aij) whose elements de- 
note the payoff or relative success of strategy i played 
against strategy j. A mixed strategy x = (sci, . . . ,x s ) is 
defined to be a probability distribution over basic strate- 
gies (i.e., Xi > for all i = 1, ...,s and Yli x i = 
Rational playing of mixed strategies will lead to a Nash 
equilibrium, that is, to a strategy x* that maximizes the 
payoff against itself This is the central concept of 
classical game theory. A more precise formulation will 
be given below. 

Game theory has been widely applied to explain the 
distribution of different phenotypes in biological popu- 
lations. These applications are based on a dynamical 
extension called evolutionary game theory, which is due 
to Maynard Smith ||. In the simplest case, the different 
phenotypes in a population are associated with the ba- 
sic strategies of a game. The time-dependent phenotype 
frequency distribution is then a mixed strategy x(i). It 
is assumed that the phenotypes are hereditary and are 
preserved under the reproduction process. The payoff of 
a basic strategy i played against the population strategy 
x(i) enters the fitness fi of the corresponding phenotype, 
that is, the expected number of viable offspring per in- 
dividual and per unit of time. This game-based popula- 
tion dynamics can be written as an evolution equation 
for the frequency distribution x(t) . It can be shown that 
every stable fixed point of this dynamics is a Nash equi- 
librium [|J . This result is conceptually important since it 
shows how strategic optimization is reached in biological 
systems through reproductive success alone, without the 
need for rational thinking. 

Classical game theory is a deterministic theory. A 
Nash equilibrium is the outcome of fully rational playing, 
without any effects of chance. Biological game dynam- 
ics describes an equally deterministic course of evolution. 
Formally, this corresponds to the limit of infinite popu- 
lations obtained by identifying the reproduction rate of 



a phenotype i with its expectation value fi. 

In this paper, we extend game theory to a stochastic 
theory including the effects of fluctuations. This is based 
on a probabilistic game dynamics suitable to describe 
the evolution of finite biological populations. A differ- 
ent form of stochastic game dynamics has been been dis- 
cussed by Marsili and Zhang in the context of economical 
systems j|. 

In biological populations, fluctuations arise since the 
actual number of viable offspring of a given individual 
in a given time interval differs from its expectation value 
determined by the fitness of the individual's phenotype. 
This may be caused by a number of different biological 
reasons. It will become clear that mechanisms produc- 
ing frequency-dependent fitness values - which are the 
subject of evolutionary game theory - also give rise to 
frequency-dependent fluctuations and thereby determine 
the probabilistic dynamics. In this formalism, the time- 
dependent population state defines a probability distri- 
bution over mixed strategies, p(x, t), and evolution takes 
the form of a Schrodinger equation in imaginary time, 
dt p(x, i) = Hp(x, t). The analogy with quantum physics 
allows for a systematic inclusion of fluctuations and sug- 
gests the name quantum game theory. 

Stochastic effects have been studied extensively in 
Kimura and Ohta's theory of neutral evolution, which 
describes the dynamics of populations whose phenotypes 
have little or no fitness difference HQ. Fluctuations are 
then the dominant force of evolution. In particular, it 
has been shown that an initially small fraction x m (t) of 
mutants in an otherwise homogeneous resident popula- 
tion can be driven either to extinction or to fixation, i.e., 
x m {t) — or x m (t) = 1 for late t. The probabilities for 
these processes depend on the fitness gap between mu- 
tants and residents and on the overall population size, 
and they determine the rate of evolution of the popula- 
tion as a whole. 

Fluctuation effects in game theory turn out to be more 
involved. Quantum game theory shows a nontrivial inter- 
play between deterministic fitness and stochastic forces, 
both of which depend on the phenotype frequencies x(t). 
Hence, a stable stationary state p*(x) - called a quantum 
Nash equilibrium - differs from its classical counterpart. 
It depends strongly on two parameters defined below, 
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the characteristic population size n* and the game cou- 
pling A. We find a crossover between neutral evolution for 
An* <C 1 and classical game theory for An* 3> 1. Fluctua- 
tions can lead to extinction or fixation; the corresponding 
probabilities now depend not only on the population size 
but also on the phenotype considered. We discuss this in 
detail for the simplest game with a mixed classical Nash 
equilibrium, the so-called hawk- dove game. 

The paper is organized as follows. In the next section, 
we define a game-based classical dynamics suitable for 
finite populations. In section 3, we discuss the 'quanti- 
zation' of this dynamics. Section 4 contains a detailed 
analysis of the quantum theory for the hawk-dove game, 
and the results are discussed in section 5. 



II. CLASSICAL GAME THEORY AND 
POPULATION DYNAMICS 

We start by recalling a few well-known definitions and 
results of classical game theory. In a game with basic 
strategies i = 1, . . . , s and relative payoffs Aij, the payoff 
of a mixed strategy x played against another mixed strat- 
egy x' is taken to be bilinear, Y^fi i=i x i^-ij x 'j = x Ax'. 
We can now define a (symmetrical) Nash equilibrium x* 
as an optimal strategy against itself, i.e., 



x*Ax* > xAx* for all strategies x. 



(1) 



Consider a population that contains the phenotypes i = 
1, . . . , s with time-dependent population numbers n(t) = 
(ni(t), . . . , n s (t)) and has total size n(t) = 
these numbers are positive integers. The evolution of this 
system can be described dctcrministically by an equation 
of the form 



1 drii 

— = fi(n) 
rii dt 



(2) 



the r.h.s. is called the fitness of the phenotype i. We 
assume its frequency dependent part is proportional to 
the payoff of the basic strategy i played against the mixed 
strategy x = n/n in a game with payoff matrix A, i.e., 
fi(n) = A(Ax)j + A', where A and A' are coefficients 
independent of x. The phenotype frequency distribution 
then obeys the closed evolution equation 



1 dxi 
Xi dt 



A/f me (x) 



with 



/r(x) = (Ax),-xAx 



(3) 



(4) 



which is well known in evolutionary game theory Q] . The 
coupling constant A < 1 describes the strength of game- 
based contributions to the fitness. Classically, it deter- 
mines only the time scale in eq. (^|) but it will have a 



crucial role in the quantum theory. The second term on 
the r.h.s. of eq. (^) ensures probability conservation, i.e., 
£\d^/dt = 0. 

This frequency dynamics is independent of population 
size. In order to include fluctuations, which depend on 
absolute population numbers, we start from the full dy- 
namics (H). We choose 



/«(n) = A/f me (x) + / size (n) 



(5) 



so that the frequencies Xi (t) follow the evolutionary game 
dynamics @ and decouple from the dynamics of the total 
population size, 



With 



1 dn 
n dt 



/ sizc (n) 



(6) 



(7) 



the latter describes standard logistic growth to a stable 
population size n*. 

This population dynamics is similar to the well-known 
Lotka-Volterra equations. It can be rewritten in terms of 
the scaled population sizes y = (yi, . . . , y s ) = n/n*, 



1 ^Vi /.game 



y% dt 



A/f mc (y/y) + f slzo (y) 



(8) 



with y = yi = n/n* . For every symmetric Nash equi- 
librium x* of the underlying game, eq. (g) has a stable 
fixed point y* = x* . 



III. QUANTUM GAME DYNAMICS 

As discussed above, the actual number of viable off- 
spring produced by an individual of phenotype i in a 
given time interval is an integer which fluctuates around 
its expectation value given by the fitness /j. In any finite 
population, these fluctuations produce deviations from 
the deterministic dynamics (||) and lead us to a stochas- 
tic description of evolution. The population state then 
becomes a propability distribution P(n, t). It is conve- 
nient to write this in the form of a quantum state, 



|P(*))=£P(M)|n) 



(9) 



with In) 



,t«i 



|0>. 



The dynamics takes the form of an (imaginary-time) 
Schrodinger equation, 



- \P{t)) = H\P{t)) 



(10) 



The Hamilton operator H contains the creation and anni- 
hilation operators a\ and dj, which obey canonical com- 
mutation relations and describe birth and death of an 
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individual of phenotype i, respectively. This formalism 
has been widely applied to related dynamical problems 
such as reaction-diffusion models; see, e.g., refs. 

In order to construct the Hamiltonian in a systematic 
way, recall the biological meaning of fitness in the clas- 
sical theory (||). The coefficient function fi(y) is not a 
simple birth rate but the effective rate of reproductive 
success, taking into account birth and death processes. 
It can be written as the difference between a birth rate 
bi(y) and a death rate dj(y) which are both positive, 

My) = h(y) - d,(y) = A(6f me (x) - df™(x)) 

+6 sizo (?i) - d sizc {n) . (If) 

Clearly, this decomposition requires additional biolog- 
ical input as will be discussed below for the hawk-dove 
game. In the quantum theory, it becomes important since 
birth and death processes have individual fluctuations. 
As operators, these rates determine the reproduction cur- 
rent, 



J t = ajaj6,-(n) - Oidj(n) 



(12) 



with n = (ni,...,n s ) — (a\ai, . . . ,a\a s ). We have 
Ji\P(t)) =£„.7'i(n,t)|n), where 

j'iOM) = mbi(n)P(n,t) - {ni + l)di(n + i)P(n + i,t) 

(13) 

is the probability current between the states |n) and |n + 
i). Here we use the shorthand 



n±i= (ni,...,ni-i,ni±l,n i+ i, 
The Hamiltonian then takes the form 



(14) 



(15) 



so that eq. (|1C| ) is equivalent to the Master equation 

(d/dt) P(n, t) = (ni - l)6j(n - i)P(n - i, t) 
-mbi(n)P(n,t) 
-mdi(n)P(n,t) 

+{m + l)d*(n + *)P(n + *, t) . (16) 

Probability conservation implies that H only has eigen- 
values with non-positive real parts. We are interested 
in particular in the leading cigenstate |<f>*(<)) with sup- 
port in the coexistence region of all phenotypes, m > 
1,. .. ,n s > 1. This eigenstate decays due to extinction 
processes rii = I — > 0. The eigenvalue E* < can 
be written in terms of the extinction current given by 
eq. ph, 



E* 

Mt) 



i=l n|ni=0 i=l n|n»=l 



With M(t) = Em>l,...,n s >l $ *( n ^)- 

For further analysis, let us approximate the Master 
equation ( plf ) by a continuous diffusion equation, which 
is conveniently written for a population state depending 
on the scaled variables y, 



with 



G i (y) = y i [b i (y) + d i (y)} 
W i (y) = y i [b i (y)-d i (y)] 



(18) 



(19) 
(20) 



Biological populations may be sufficiently large so that 
their variation in relative size, which is of order 



(21) 



can be neglected. The fluctuations in phenotype compo- 
sition may still be significant, depending on the scaled 
game coupling constant An*. They are described by a 
projected diffusion equation for the phenotype state 



(x,t) = J dy$(y,t)5(x-y/y) . 



(22) 



Long-term fluctuations are characterized by the leading 
eigenstate (p* (x, t) with support in the coexistence region 
< Xi < 1 and the corresponding eigenvalue e* < 0. Re- 
moving the time-dependence by normalization defines a 
stationary phenotype probability distribution p*(x), the 
quantum Nash equilibrium. For game-based evolution, 
the diffusion coefficients and the drift fields have a non- 
trivial dependence on the phenotype composition x, pre- 
venting a solution in closed form. Therefore, we will now 
turn to a specific example. 



IV. THE QUANTUM HAWK-DOVE GAME 

The hawk-dove game [|| is one of the simplest classical 
games. 'Hawks' (i = 1) and 'doves' (i = 2) are the two 
phenotypes of this game, and there is a single indepen- 
dent frequency variable x = xx = 1 — X2- The game is 
defined by the (suitably normalized) payoff matrix 



1-c 2 
1 



(23) 



with a constant c > 1, leading to a classical population 
dynamics of the form (^) , (0), 



(17) 



Ax 
dt 



Xx(l — x)(l — cx) 



(24) 
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The unique stable fixed point is the Nash equilibrium 
x* = l/c. 

The hawk-dove game dynamics has been used to ex- 
plain instinctive agression control in animal species. In- 
dividuals of the same species may fight for territory, 
mating partners, etc. 'Hawks' are aggressive individuals 
who escalate fights, while 'doves' avoid escalation and 
retreat. Hence, a hawk will always win against a dove 
{A12 = 2,A 2 i = 0), compared to the payoff of doves 
againts doves (A22 = 1). However, hawk- hawk encoun- 
ters involve a fitness decrease due to mutually inflicted 
injuries (An = 1 — c). In species with heavy weaponry 
(c ^> 1), fights are usually limited to ritual display of 
force and truly aggressive individuals are rare, as indi- 
cated by the Nash equilibrium x* < 1. Needless to say, 
this is but the simplest approximation to the complexities 
of animal behavior. 

The biological interpretation also suggests a decompo- 
sition ( pd| ) of the fitness into birth and death processes. 
Here we choose 



6f mo (x) = (Bx) 4 xDx 
df mo (x) = (Dx),-xBx 



with 



B 



1 2 
1 



D 



c 




(25) 



(26) 



The matrix B describes game-related variations of the 
birth rates, while D contains the deaths due to fights. 
The offset terms xBx and xDx required by eq. (J|) are 
associated with the other type of process, respectively. 
The population size is assumed to be controlled by a bare 
birth rate and a variable death rate, 



6 sizc = 1 , d s izc = y 



(27) 



Of course, this decomposition is not unique, and a dif- 
ferent biological context may suggest a different choice. 
The qualitative results described below do not depend on 
these details. 

Using the decomposition (^)-(^), we obtain the one- 
dimensional phenotype diffusion equation 

J^(*. t) = JLiLjrtaMz, t) - \JLv(xMx, t) (28) 



with 



g(x) = x(l - x)[(l - x)(bi(x) + di(x)) 

+x(b 2 (x) +d 2 (x))] (29) 



and 



v(x) = x(l — x)(l — x/x*) 



(30) 



The quantum Nash equilibrium p*(x) and the eigen- 
value e* can be computed approximately. For A = 0, 



the solution is p*(x) = 1 and e* = l/2n*, describ- 
ing the well-known case of neutral evolution. Correc- 
tions for An* <C 1 have the form of a power series in 
An*. For An* 1, obtain the approximate functional 
functional form of the solution by neglecting extinctions, 
i.e., by setting e* ~ 0. The current in the Nash state, 
j*(x) = (l/2n*)(A/Ax)(g(x)p*(x)) - \v(x)p*(x), is also 
zero in this approximation, yielding 



p*(x)=p*(x*) 



g(s*) 

g(x) 



exp 



2An* 



dx'^1 

g(x') 



(31) 



For very large populations, p*(x) is seen to be a Gaussian 
of width ~ 1/ \J An* centered around the classical Nash 
equilibrium x* . The approximation ( |3l"l) is seen to be 
self-consistent for 



1 S'(0) 



1 <?'(!) 



" 111111 ' •> : \ /,,yv ~ x ~ x max — 1 <-»*% \ • (32) 

Of course, the current cannot be neglected close to the 
boundary, where we have j*(x) = (-l/2n*)g' (0)(f>(0) + 
0(x) and j*(x) = (-l/2n* V(lWl)+0(l-a;). Match- 
ing the two regimes and using (17) gives an asymptotic 
expression for e*. The ratio of the extinction currents 
takes the simple form 



-j*(o) 



exp 



2Ar 



da; 



,v(x>) 



g(x') 



(33) 



Fig. 1 shows the quantum Nash state p* (x) obtained nu- 
merically for various values of the scaled game coupling 
An*. The solution of the full Master equation ( pii|) is 
projected onto the frequency variable x using eq. (\22\). 
The crossover from neutral to game-dominated behavior 
is seen already at small equilibrium population sizes (here 
n* = 70). The strong-coupling approximation (pn]), also 
shown in fig. 1, turns out to be an excellent approxima- 
tion in the regime An*>l, £ min (A, n*) < x < x max (A, n*). 




Fig. 1. The quantum Nash equilibrium p*(x) for n* — 70 
and x* = 0.28. Numerical solution of the Master equation 
@ for A = 1,0.1,0.01 (dots), the limit A = of neutral 
evolution (dashed line), and the strong-coupling solution ( |3l| ) 
for x min (A,n*)<x<a; max (A,n*) (solid lines). 
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A comprehensive analytical treatment of the quantum 
hawk-dove game is beyond the scope of this paper. In- 
stead we now discuss properties of the solution which are 
valid beyond this specific example. 

V. RESULTS AND DISCUSSION 

The quantum Nash equilibrium p*{x) describes the 
likelihood of finding phenotype frequencies x in long-term 
observations of biological sample populations. The fluc- 
tuations in phenotype composition depend on the scaled 
game coupling constant Xn* and may thus be significant 
even in large populations where the relative size fluctua- 
tions can be neglected. There are two modes of evolution. 

In the weak-coupling regime Xn* <C 1, the system is 
dominated by stochastic forces. These produce a broad 
quantum Nash equilibrium, which implies large frequency 
variations, 

<(x-(x)) 2 )~I. (34) 

In the limit A = 0, the evolution becomes strictly neutral. 

In the strong- coupling regime An* ^> 1, the game- 
related deterministic forces become relevant. These sys- 
tems have only small frequency variations around a clas- 
sical Nash equilibrium, 

((x-x*) 2 )~l/An*, (35) 

as shown by the quantum Nash state (|3l]) for the hawk- 
dove game. Classical game theory is recovered in the 
limit n* — > oo. 

At any finite Xn* , the mixed Nash equilibrium can be 
altered drastically by the extinction of phenotypes. The 
probability of extinction depends strongly on the pheno- 
type considered. In the hawk-dove game, hawks face a 
much higher risk of extinction than doves, as indicated by 
the ratio of the extinction currents in the quantum Nash 
state given by (^). For x* < 1/2, the current j*(0) is 
exponentially larger in magnitude than This result 

also implies that an initially small hawk mutant popula- 
tion in a dove resident population is less likely to grow to 
its classical equilibrium frequency x* than a dove mutant 
in a hawk resident population. 

In a similar way, stochastic effects may influence the 
internal evolution of phenotypes even in the strong- 
coupling regime, where the overall frequencies are close 
to the classical Nash equilibrium x*. In the hawk-dove 
game, consider a mutant phenotype with a linkage dis- 
equilibrium such that it can invade only the hawk sub- 
population. According to the standard theory of neutral 
evolution, the fixation probability of this mutant depends 
on the fitness difference to the resident hawks and on the 
effective hawk population size n® s . It is easy to show that 
this may be much smaller than the effective population 
size rij 9 for doves, as given by the ratio 



nf = x* b 2 {x*) + d 2 {x*) 

nf 1-x* bi{x*) + di(x*) ' [ ' 

Hence, hawks face a larger mutation load and a lesser 
chance of fixation for adaptive mutations than doves. 

To summarize: In classical game theory, the basic 
strategies represented in a mixed Nash equilibrium x* 
are equivalent in the sense that their payoffs (Ax)j are 
all equal to the average payoff xAx, see eq. (||) . Stochas- 
tic effects break this equivalence and alter the equilib- 
rium state. In biological systems, stochasticity is caused 
by frequency-dependent birth and death rates, that is, 
by the same mechanism that gives rise to frequency- 
dependent fitness values and underlies the application of 
classical game theory. Stochastic evolution creates a bias 
against phenotypes with larger fluctuations in their birth 
and death rates. 

Quantum game theory opens a systematic way to 
quantify these stochastic effects. It contains Maynard- 
Smith's evolutionary game theory and Kimura's theory 
of neutral evolution as the limit cases of weak and strong 
stochasticity, respectively. We have introduced here the 
relevant concepts and applied them to the simplest kind 
of game. Obvious extensions include more complicated 
games such as bimatrix games. 

Another avenue for future work is cooperative fluctu- 
ations in phenotype and space. In biological systems, 
population states p(x, r) now depend also on spatial co- 
ordinates, and stochastic game theory becomes a non- 
equilibrium quantum field theory. Population dynami- 
cal problems involving diffusion and migration have been 
treated by field-theoretic methods (l(J , and more recently 
the process of biological speciation has been identified 
as a coupled phase separation in phenotype and real 
space pT| . 

Finally, it is tempting to speculate about a different 
kind of randomness in game dynamics. The stochastic 
effects discussed so far originate from events random in 
time. However, we may also consider games with a payoff 
random for individual pairings of players but indepen- 
dent of time. This quenched disorder may affect Nash 
equilibria in new ways. 
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